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WHAT IS CL AIMED IS: 

1. A method comprising: 

receiving a ranked list of search results from a search engine based on a search 
query; and 

estimating a relevance value of a particular search result in the ranked list based 
on its rank and actual relevance values and ranks of at least two others of 
the search results. 

2. The method of claim 1 wherein said estimating comprises fitting a curve, to 
represent relevance as a function of rank, to the actual relevance values and the ranks of 
the at least two others of the search results. 

3. The method of claim 2 wherein said estimating further comprises evaluating 
the curve at the rank of the particular search result to estimate the relevance value. 

4. The method of claim 3 wherein the curve is a line. 

5. The method of claim 1 wherein said estimating comprises determining an 
interpolation function, to represent relevance as a function of rank, for the actual 
relevance values and the ranks of the at least two others of the search results. 

6. The method of claim 5 wherein said estimating further comprises evaluating 
the interpolation function at the rank of the particular search result to estimate the 
relevance value. 

7. The method of claim 1 wherein said estimating comprises linearly 
interpolating between two actual relevance values whose ranks bracket the rank of the 
particular search result. 

8. The method of claim 1 wherein the actual relevance values are supplied by the 
search engine. 
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9. The method of claim 1 wherein the actual relevance values are not supplied by 
the search engine. 

10. The method of claim 9 further comprising: 

determining a first actual relevance value for a most-relevant one of the search 
results; and 

determining a second actual relevance value for a least-relevant one of the search 
results; 

wherein said estimating comprises linearly interpolating between the first actual 
relevance value and the second actual relevance value. 

1 1 . A method of weighting search results from a search engine based on a search 

query, the method comprising: 

determining a plurality of categories associated with the search query; 
for each of the categories, determining an associated category weighting value for 
the search engine; 

determining a first associated relevance value for each of the categories based on 
the search query and one or more query terms associated with the 
category; and 

determining a weighting value based on the category weighting values and the 
first associated relevance values. 



12. The method of claim 1 1 further comprising: 

determining a second associated relevance value for each of the categories by 
dividing its first associated relevance value by a sum of all of the first 
associated relevance values. 

13. The method of claim 12 wherein said determining the weighting value 
comprises determining the weighting value based on a sum, over the categories, of each 
product of the associated category weighting value and the second associated relevance 
value. 
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14. A method comprising: 

submitting a search query to a plurality of search engines; 

receiving, from each of the plurality of search engines, an associated ranked list of 

search results based on the search query; 
receiving a plurality of actual relevance values for a plurality of the search results 

based on the search query; 
for at least one of the search results absent the actual relevance value, estimating 

its relevance value based on its rank, and the ranks and the actual 

relevance values of at least two others of the search results; 
determining, for each of the plurality of search engines, an associated weighting 

value; 

determining, for each of the ranked lists, an associated weighted relevance value 
for each of its search results based on the estimated relevance value or the 
actual relevance value of the search result and the weighting value 
associated with the search engine that provided the ranked list; 

combining the ranked lists into a single list; and 

sorting the search results in the single list based on the associated weighted 
relevance values. 

15. The method of claim 14 wherein the actual values comprise normalized, 
search-engine-supplied relevance values. 
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16. The method of claim 14 wherein said determining the associated weighting 
value for a search engine comprises: 

determining a plurality of categories associated with the search query; 
determining an associated category search engine weighting value for each of the 
categories; 

determining a first associated relevance value for each of the categories based on 
the search query and one or more query terms associated with the 
category; 

determining a second associated relevance value for each of the categories by 

dividing its first associated relevance value by a sum of all first associated 

relevance values; and 
determining the associated weighting value based on a sum, over the categories, 

of each product of the associated category search engine weighting value 

and the second associated relevance value. 

17. The method of claim 14 wherein said estimating comprises: 

fitting a curve, to represent relevance as a function of rank, to the actual relevance 
values and the ranks of the at least two others of the search results; and 

evaluating the curve at the rank of the particular search result to estimate the 
relevance value. 

18. The method of claim 14 wherein said estimating comprises: 
determining an interpolation function, to represent relevance as a function of rank 

for the actual relevance values and the ranks of the at least two others of 
the search results; and 
evaluating the interpolation function at the rank of the particular search result to 
estimate the relevance value. 

1 9. The method of claim 1 4 wherein the actual relevance values are search- 
engine-supplied. 
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20. The method of claim 14 wherein the actual relevance values are not search- 
engine-supplied. 

21. An apparatus comprising: 

a computer programmed to perform acts of: 

receiving a ranked list of search results from a search engine based on a search 
query; and 

estimating a relevance value of a particular search result in the ranked list based 
on its rank and actual relevance values and ranks of at least two others of 
the search results. 

22. The apparatus of claim 21 wherein said estimating comprises fitting a curve, 
to represent relevance as a function of rank, to the actual relevance values and the ranks 
of the at least two others of the search results. 

23. The apparatus of claim 22 wherein said estimating further comprises 
evaluating the curve at the rank of the particular search result to estimate the relevance 
value. 

24. The apparatus of claim 23 wherein the curve is a line. 

25. The apparatus of claim 21 wherein said estimating comprises determining an 
interpolation function, to represent relevance as a function of rank, for the actual 
relevance values and the ranks of the at least two others of the search results. 

26. The apparatus of claim 25 wherein said estimating further comprises 
evaluating the interpolation function at the rank of the particular search result to estimate 
the relevance value. 

27. The apparatus of claim 21 wherein said estimating comprises linearly 
interpolating between two actual relevance values whose ranks bracket the rank of the 
particular search result. 
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28. The apparatus of claim 21 wherein the actual relevance values are supplied 
by the search engine. 

29. The apparatus of claim 21 wherein the actual relevance values are not 
supplied by the search engine. 

30. The apparatus of claim 29 wherein the computer is programmed to perform 
further acts of: 

determining a first actual relevance value for a most-relevant one of the search 
results; and 

determining a second actual relevance value for a least-relevant one of the search 
results; 

wherein said estimating comprises linearly interpolating between the first actual 
relevance value and the second actual relevance value. 

31 . An apparatus for weighting search results from a search engine based on a 
search query, the apparatus comprising: 

a computer programmed to perform acts of: 

determining a plurality of categories associated with the search query ; 
for each of the categories, determining an associated category weighting value for 
the search engine; 

determining a first associated relevance value for each of the categories based on 
the search query and one or more query terms associated with the 
category; and 

determining a weighting value based on the category weighting values and the 
first associated relevance values. 
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32. The apparatus of claim 31 wherein the computer is programmed to perform a 
further act of: 

determining a second associated relevance value for each of the categories by 

dividing its first associated relevance value by a sum of the first associated 
relevance values. 

33. The apparatus of claim 32 wherein said determining the weighting value 
comprises determining the weighting value based on a sum, over the categories, of each 
product of the associated category weighting value and the second associated relevance 
value. 

34. An apparatus comprising: 

a computer programmed to perform acts of: 

submitting a search query to a plurality of search engines; 

receiving, from each of the plurality of search engines, an associated ranked list of 

search results based on the search query; 
receiving a plurality of actual relevance values for a plurality of the search results 

based on the search query; 
for at least one of the search results absent an actual relevance value, estimating 

its relevance value based on its rank, and the ranks and the actual 

relevance values of at least two others of the search results; 
determining, for each of the plurality of search engines, an associated weighting 

value; 

determining, for each of the ranked lists, an associated weighted relevance value 
for each of its search results based on the estimated relevance value or the 
actual relevance value of the search result and the weighting value 
associated with the search engine that provided the ranked list; 

combining the ranked lists into a single list; and 

sorting the search results in the single list based on the associated weighted 
relevance values. 
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35. The apparatus of claim 34 wherein the actual values comprise normalized, 
scarch-engine-supplied relevance values. 

36. The apparatus of claim 34 wherein said determining the associated weighting 
value for a search engine comprises: 

determining a plurality of categories associated with the search query; 
determining an associated category search engine weighting value for each of the 
categories; 

determining a first associated relevance value for each of the categories based on 
the search query and one or more query terms associated with the 
category; 

determining a second associated relevance value for each of the categories by 

dividing its first associated relevance value by a sum of all first associated 

relevance values; and 
determining the associated weighting value based on a sum, over the categories, 

of each product of the associated category search engine weighting value 

and the second associated relevance value. 

37. The apparatus of claim 34 wherein said estimating comprises: 

fitting a curve, to represent relevance as a function of rank, to the actual relevance 
values and the ranks of the at least two others of the search results; and 

evaluating the curve at the rank of the particular search result to estimate the 
relevance value. 

38. The apparatus of claim 34 wherein said estimating comprises: 
determining an interpolation function, to represent relevance as a function of rank, 

for the actual relevance values and the ranks of the at least two others of 
the search results; and 
evaluating the interpolation function at the rank of the particular search result to 
estimate the relevance value. 
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39. The apparatus of claim 34 wherein the actual relevance values are search- 
engine-supplied. 

40. The apparatus of claim 34 wherein the actual relevance values are not search- 
engine-supplied. 

41. An article comprising: 

a computer-readable medium having computer-readable program code to cause a 

computer to perform acts of: 
receiving a ranked list of search results from a search engine based on a search 

query; and 

estimating a relevance value of a particular search result in the ranked list based 
on its rank and actual relevance values and ranks of at least two others of 
the search results. 

42. The article of claim 41 wherein said estimating comprises fitting a curve, to 
represent relevance as a function of rank, to the actual relevance values and the ranks of 
the at least two others of the search results. 

43. The article of claim 42 wherein said estimating further comprises evaluating 
the curve at the rank of the particular search result to estimate the relevance value. 

44. The article of claim 43 wherein the curve is a line. 

45. The article of claim 41 wherein said estimating comprises determining an 
interpolation function, to represent relevance as a function of rank, for the actual 
relevance values and the ranks of the at least two others of the search results. 

46. The article of claim 45 wherein said estimating further comprises evaluating 
the interpolation function at the rank of the particular search result to estimate the 
relevance value. 

- 18- 

IO.WSSOOJ82 |>otcft( Applicnliou.rioc 



05/14/2004 13:04 5123275452 



TOLER LARSON & ABEL PAGE 25/33 

Attorney Dockel No.: . J3^SS00382 



47. The article of claim 41 wherein said estimating comprises linearly 
interpolating between two actual relevance values whose ranks bracket the rank of the 
particular search result. 

48. The article of claim 41 wherein the actual relevance values are supplied by the 
search engine. 

49. The article of claim 41 wherein the actual relevance values are not supplied 
by the search engine. 

50. The article of claim 49 wherein the computer-readable program code further 

causes the computer to perform acts of: 

determining a first actual relevance value for a most-relevant one of the search 

results; and 

determining a second actual relevance value for a least-relevant one of the search 
results; 

wherein said estimating comprises linearly interpolating between the first actual 
relevance value and the second actual relevance value. 

5 1 . An article for weighting search results from a search engine based on a search 

query, the article comprising: 

a computer-readable medium having computer-readable program code to cause a 

computer to perform acts of: 
determining a plurality of categories associated with the search query; 
for each of the categories, determining an associated category weighting value for 

the search engine; 

determining a first associated relevance value for each of the categories based on 
the search query and one or more query terms associated with the 
category; and 

determining a weighting value based on the category weighting values and the 
first associated relevance values. 
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52. The article of claim 51 wherein the computer-readable program code further 
causes the computer to perform an act of: 

determining a second associated relevance value for each of the categories by 

dividing its first associated relevance value by a sum of all first associated 
relevance values. 

53. The article of claim 52 wherein said determining the weighting value 
comprises determining the weighting value based on a sum, over the categories, of each 
product of the associated category weighting value and the second associated relevance 
value. 



54. An article comprising: 

a computer-readable medium having computer-readable program code to cause a 

computer to perform acts of: 
submitting a search query to a plurality of search engines; 
receiving, from each of the search engines, an associated ranked list of search 

results based on the search query; 
receiving a plurality of actual relevance values for a plurality of the search results 

based on the search query; 
for at least one of the plurality of search results absent an actual relevance value, 

estimating its relevance value based on its rank, and the ranks and the 

actual relevance values of at least two others of the search results; 
determining, for each of the plurality of search engines, an associated weighting 

value; 

determining, for each of the ranked lists, an associated weighted relevance value 
for each of its search results based on the estimated relevance value or the 
actual relevance value of the search result and the weighting value 
associated with the search engine that provided the ranked list; 

combining the ranked lists into a single list; and 

sorting the search results in the single list based on the associated weighted 
relevance values. 
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55. The article of claim 54 wherein the actual relevance values comprise 
normalized, search-engine-supplied relevance values. 

56. The article of claim 54 wherein said determining the associated weighting 
value for a search engine comprises: 

determining a plurality of categories associated with the search query; 
determining an associated category search engine weighting value for each of the 
categories; 

determining a first associated relevance value for each of the categories based on 
the search query and one or more query terms associated with the 
category; 

determining a second associated relevance value for each of the categories by 

dividing its first associated relevance value by a sum of all first associated 

relevance values; and 
determining the associated weighting value based on a sum, over the categories, 

of each product of the associated category search engine weighting value 

and the second associated relevance value. 

57. The article of claim 54 wherein said estimating comprises: 

fitting a curve, to represent relevance as a Junction of rank, to the actual relevance 
values and the ranks of the at least two others of the search results; and 

evaluating the curve at the rank of the particular search result to estimate the 
relevance value. 

58. The article of claim 54 wherein said estimating comprises: 

determining an interpolation function, to represent relevance as a function of rank, 
for the actual relevance values and the ranks of the at least two others of 
the search results; and 

evaluating the interpolation function at the rank of the particular search result to 
estimate the relevance value. 
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59. The article of claim 54 wherein the actual relevance values are search-engine- 
supplied. 

60. The article of claim 54 wherein the actual relevance values are not search- 
engine-supplied. 



I0.1.1-SSOO382 UxtAW Appticolfon.doc 



-22- 



